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ABSTRACT 



Suggesting that, James Coleman's paper on massive 
school desegragation reveals methodological flaws of such magnitude 
that they raise serious questions as to the validity of the 
conclusions, this paper addresses a full - sequence of perceived 
methodological* errors found in' the Coleman document, but- does not 
dismiss the conclusions based upon initial errors., no matter how 
cogent they be. This approach makes the implicit assumption that, at 
each step, all previous operations upon the data are both well 
reasoned and technically correct. As the critique progresses, it 
becomes obvious to what extent this assumption is incorrect. The 
first point made concerns the data base' used, and, by implication, 
the range of variables considered for inclusion in the various 
regression models used. The second point relates to the unit of 
analysis employed. Another potential problem concerns the 
partitioning of the data into two categories based upon school 
district size. The treatment (or lack of treatment) of the standard 
errors of. the regression coefficients is also criticized. Finally, 
one of the more misleading of the methodological errors is reflected 
in the attempt to control for the unjique characteristics of ,the 
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In numerous press conferences, interviews, Congressional hearings and, 
m most recently,, in the pages of the Phi Delta Kappan fDr # James Coleman has made . 
^ . the rather forceful statement that his research shows massive school desegre- 
^ gation efforts to be self-defeating. It is his position that such attempts 3 simply 
q hasten the departure of white children. from the public schools of the involved 
districts. While such statements have struck a responsive chord in a '"large 
section of Americans^ contrary to his claim they are not very well grounded in 
* empirical facts. *The analyses that have, until recently, been unavailable fo,£he 
gerferal public and the research^ coirmunity are now set forth in "Tr&ncis in School 
Segregation, 1968-73". A careful /perusal of this document reveals to the reader 
methodological flaws of sufficient magnitude to raise- serious questidhs^as to 
the validity of the conclusions. - * : . 

What now follows represents tbis author's attempt to offer a critique of 
the above mentioned paper. As such, no claims are%made to its completeness; 
there could wefl be errors of omission or cormrission which would serve to 
further weaken the inferential chain welded'by Coleman al_. What is done by 
the author .attempts to address a full sequence of perceived methodological 
errors rather than dismissing the conclusions based upon initial errors 00 
matter how cogent. This approach, there fore * makes the implicit assumption 
T-i that, at each step, all previous operations upon the data have been both well 
» ^reasoned and technically correct. As the critique progresses it will become 
painfully obvious to .what extent this assumption is] in fact, incorrect. 
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-• The first point' which needs be made -concerns the data base used byj:he . 

* authors and, by implication, the range of variables considered for inclusion . 
in their various regression models. Their sole source* of "information appears 
to have been data supplied by school districts to the Department of Health, 
Education and Welfare. The variables on which information was supplied relate, 
solely to, public school attendance:,, total number oj studelfifs, total white, ' 
total black, their respective, proportions^ fn the various districts and schools, 
and'the changes taking place in them from year .to year. While, .in and of them- 
selves, they present an interesting view of changes taking place in public edu- 
cation, they cannot be 'relied -upon to furnish : 'us with the degree of under- 

' standing necessary for us to address the causes' of decreasing white enrollment. 
Even if we are to make the assumption, as do Coleman et aj_, that this 
phenomenon -reflects" white residential relocation ("white flight"), we'must ask 
ourselves what variables woufd likely be related "to such an activity. Likely 
sets of variables would, of necessity, include factors related .to economics, 
demographics,' crime and violence, and the quality of the environment to name but 
a few. "While public school desegregation deserves its place as an educational 
factor potentially affecting white' school attendance focusing upon such, a 
phenomenon' as the primary predictive variable would appear to be both short- 
sighted and highly likely to lead to incorrect inferences about the true nature 
of the problem., 

.-JThe second methodological point relates to the unit of analysis employed 
by Coleman ert aU Based on what is reported as the number of observations 
arid degrees of freedom one must assume that the basic unit of analysis is each 
year's data for each city, (or the between year 's -^differences to be more 
accurate). Apparently the roughly 5 between -adjacent-pairs -of -years differences 
for each city are merely thrown together for use in the regression analysis 
with the specific cf t y identification completely disregarded. ' 
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This would seem to be completely inappropriate. A more reasonable procedure 
vjould have been to have used each- city as the unit of analysis.-, forming^ some 
measure of relationship between change in white attendance and change in se- 
gregation within each city based on the time points for that city alone. 
-These could then have been averaged across all cities with approp/iate confi- 
dence intervals established .based on correct degrees af freedom/ 

Yet another questionable approach is the pooling of data from the various 
years to predict the proportion change in white attendance.- This, would seem to 
be inappropriate in that individual years are disregarded .with data arising* . 
from one pair of years being considered equally with data arising frofn any 
other pair of years. Since it would seem- that "the. author's intentions were to 

investigate- trends* in school segregation, such an approach ignores a key 

^ I % ' 

variable in.their study. /• c 

Another potential problem has to do with the partitioning of the data 
into two categories based upon school district size. Coleman e_t al_ justify 
this partitioning "because of the indications that response to desegregation 
differs considerably .in the very large cities from the response in t'h'e" smaller 
ones". It would have been quite easy for such an assumption to have been 
statistically tested by pooling all data, doing : the- regression, analysis, then 
introducing a durrniy variable to' see if a significant gain jn the R 2 was pro - 
duced. If not, the coefficients could then have;t>een estimated with smaller, 
standard error due to the larger number of degrees of freedom for error. 

An even more disturbing practice has to. do with their treatment (or lack 
of treatment) of the standard errors of their regression" coefficients. In 
most instances the authors failed to temper substantiveinterpretation of re- 
gression coefficients in the light Of their standard errors. Where standard 

errors- are sufficiently large as to 'make tenable the null hypothesis that. a 

A • " • 
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particular regression coefficient is not different from zero, the authors • 
Should have eliminated the variable from the equation,- rerun the, analysis, and' 
based .subsequent use .of the equation on the more parsimonious model . " In Table 
14 and the use of the results displayed, there is a glaring example of this 
.problem since many of the coefficients are not substantially larger than their 
standard errors, (or are, in fact,- smaller). In producing their .estimac:\ for 
various changes in their dependent measures they agajn failed to take into 
account the standard errors of their coefficients. Had confidence intervals 
been :> generated_about. their estimated loss of white students, "it seems in- ' 
evitable that 0% loss would be included'."^ 

«... •. . • . ... N 

A fairly minbr error'of omission takes place in the author's caution : • 
with respecfto the interpretation' of the regression coefficients presented in 
Table 1*4. The authors state' that there is some difficulty in interpreting the 
actual coefficients in the regression equations citing problems associated 
•with multi-cpliniarity. While this may be true, the authors should present em- 
pirical' evidence in the form of a correlation matrix of the variables to make 

• such an interpretation more plausible as well as to give the reader more infor- 
mation about what is actually taking place in the data. 

In arriving at estimates for regression coefficients discussed later on, 
the authors state that they averaged" the first three coefficients common to 
four time-lagged regression models presented earlier. Such an averaging pro- 
cedure is completely unjustifiable. For each model the first three estimates 
are conditioned upon the inclusion of the additional terms, and unless the 
first three terms are statistically independent' of all subsequent terms, they 
do not estimate the same quantity. Their statement that "'such an analysis 

cannot provide a conclusive answer is grossly understated in the light of their 

erroneous methods. 
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One of the more misleading of the methodological errors is reflected in 
their attempt to control for the unique characteristics of the cities through 
the use of dummy variables. Even if the criticismsNmentioned with respect to 
the unit of analysis are ignored (and they must be to ahlw for this .particular 
analysis), a very stringent assumption must be met if the author's use of dummy 
vanables^for cities' is to do as much for this analysis as they Seem to imply. 
Since the values of the dummy variables are constant across all six years for . 
each individual city, we must assume that the values for all other possibW re- 
levant variables, remain constant. This .assumption seems incredibly farfetcheo\ 
on its face. In addition, the authors present no empirical evidence which 
would justify it. Clearly, these cities have not only their own particular, 
rate of economic activity, crime, etc., but their own particular year-to-year 

changes in these rates - change which could' be related as strongly to. changes 

/ .... 

in white school attendance as are changes in the segregation index.. 

Finally, in discussing predicted white school attendance across time, the 
authors chose to focus upon predictions arising from Equation 3 rather than 
those from Equation 1 (See Table 14).. Even here, the authors fail to provide 
confidence intervals on their estimates, a procedure which would seem especially 
important since four of the eight variables, in equation 3 have coefficients less 
than, or only slightly larger than, their standard .errors . In the Tight of the 
instabilities likely to result from such a situation, it would seem more ap-- 
propriate to focus upon the projections based on equation 1 - projections which* 
show a difference of only 3% with desegregation compared .to no desegregation. * 

Thus it may be seen -tlxat the work reported by Coleman et al^ suffers from a 
sequence of serious methodological flaws, Their failure to replicate the find- • 
ings of other researchers more equivocal with respect to the impact of desegre- 
gation upon white enrollment may be due, in large part, to the afore mentioned 
problems. While no firm statemeat 'can be made to the accuracy of their coftclu- 
'sions, one thing which can be stated is that they have wandered far from their 
original data in coming to. those conclusions. 
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Table 14. REGRESSION COEFFICIENTS FOR ANALYSES OF 
WHITE STUDS'Tf LOSS . TO CENTRAL CITIES 



Equation 1 
&R 

■ Prop, black 
In U 

Constant 



L'unber of observations 



Largest 21 



.277(.062) 
-*135(.028) 
~7001T.O"OT) 
.003 



f 



.28 
(103) 



.Next 46 



: .091C..031-) 
-.086 (.151) 
-.047(.011) 
.503 .. ^ 

.24 
(239) 



Including inter-district segregation in SMSA, and interaction of desegregation 
with- South • 



Equation 2 

AR . 

Prop, black 
In N 
.R SMSA 
ARxS 
Constant 



.195(.158) 
.047 (.041) 
.007 (.008) 
.162(.052) 
.144 (.172) 
.064 

.35 



.008(.151) 
-.033(.019) 
-.042(.0li) 
-.109 (.024) 

.122(.152) 

.450 

.29 



Including interactions of desegregation with proportion black and inter-district 
segregation, and also including South as dunmy variable / 



Equation 3 

•-» 

Proportion black 
' In !l 

R SMSA 
A RxSouth 
" A RxProp. black 

South 
Constant 

R 2 



-.4*60(.187) 
.050 (.037) 
.003 (.007) 

-.208(.045) 
.146(.201) 

1.774(.313) 
.544 (.501) 

-.006(.010) 
-J037 

.60 » 



-.147 (.173)' 

-.027(.022) 

-.04K.011) 

-.10K.029) 
.108(.165) 
.406 (.254) 
.664(.385) 

-.00K.007) 
.43337 

.32 



Reproduced from "Trends in School Segregation, 1958-73" - page 59 



